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Abstract 

We  present  dual-band  infrared  image  data  collected  as  part  of  the  Multidomain 
Smart  Sensors  effort  at  the  U.  S.  Army  Research  Laboratory  (ARL).  The  goal  of  this  ef¬ 
fort  is  to  produce  large-format,  staring  focal  plane  arrays  (FPAs)  that  are  able  to  see  the 
battlefield  in  both  the  3  to  5 -pm  (mid- wave  infrared  or  MWIR)  and  8  to  12-|Lim  (long¬ 
wave  infrared  or  LWIR)  atmospheric  transmission  windows.  The  data  was  collected  un¬ 
der  laboratory  and  field  test  conditions  using  a  simultaneously  integrating,  pixel- 
registered,  256  X  256,  dual-band  focal  plane  array  produced  by  Lockheed  Martin  using 
quantum-well  infrared  photodetector  (QWIP)  technology.  The  dual-band  FPA  was  in¬ 
stalled  in  a  camera  that  was  taken  in  the  field  to  gather  image  data  on  military  targets. 

The  pixel-registered  dual-band  FPA  is  well  suited  for  the  application  of  image  fusion  al¬ 
gorithms.  We  also  applied  some  of  these  image  fusion  techniques  to  the  imagery  to  en¬ 
hance  the  visibility  of  targets. 


1.0  INTRODUCTION 

Over  the  last  few  years,  much  effort  has  been  invested  in  producing  dual-band  infrared  (IR)  im¬ 
aging.  Some  of  the  motivations  behind  such  efforts  are  enhanced  detection  of  targets  in  clutter,  the  ability 
to  distinguish  between  targets  and  decoys,  and  remote  absolute  temperature  measurement.  These  efforts 
have  produced  infrared  focal  plane  arrays  (FPAs)  operating  at  two  wavelengths  in  the  mid-wave  infrared^ 
(MWIR,  3  to  5  pm)  and  FPAs  operating  at  two  wavelengths  in  the  long-wave  infrared^  (LWIR,  8  to  12 
pm)  and  also  FPAs  operating  in  both  the  LWIR  and  MWIR^  spectral  regions.  In  earlier  efforts,  the  pixels 
of  the  individual  colors  were  either  not  spatially  registered  or  the  outputs  were  read  out  sequentially  (not 
simultaneously). 


^J.  Caufield,  et  al,  “Simultaneous  Integrating  Staring  Two  Color  IRFPAs,”  Proceedings  of  the  1997  IRIS  Detector 
Specialty  Group  Meeting  on  Detectors,  p.  351,  ERIM  (1998). 

^  M.  Sundaram,  et  al,  “Advances  in  QWIP  FPA  (2-Color  and  1-Color)  Technology,”  Proceedings  of  the  1998  IRIS 
Specialty  Group  Meeting  on  Detectors,  p.  219,  ERIM  (1999). 

^  D.  Scribner,  J.  Schuler,  P.  Warren,  M.  Satyshur,  M.  Kruer,  “Infrared  Color  Vision:  Separating  Objects  from  Back¬ 
grounds,”  Proceedings  of  the  SPIE  -  Infrared  Detectors  and  Focal  Plane  Arrays  V,  Vol.  3379,  p.  2  (1998). 
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Recently,  the  Army  Research  Laboratory  (ARL),  in  federation  with  several  industry  and  aca¬ 
demic  partners,  has  developed  the  concept  of  a  multi-domain  smart  sensor  (MDSS)/  This  conceptual  or 
"technology  demonstration"  system  is  envisioned  as  a  single  unit  combining  both  passive  and  active  sen¬ 
sor  components  with  advanced  signal  processing  and  aided  target  recognition  (ATR)  tools.  Such  a  sensor 
would  enhance  situational  awareness  on  the  battlefield  in  all  ambient  conditions  by  locating  and  classify¬ 
ing  threats  with  increased  effectiveness  over  existing  systems.  An  integral  part  of  the  MDSS  system  is  a 
dual-band  infrared  imager  operating  in  the  MWIR  and  LWIR  bands.  This  imager  would  be  advantageous 
over  single-color  IR  cameras  (either  LWIR  or  MWIR)  because  it  can  operate  in  a  wider  range  of  ambient 
conditions  and  be  more  effective  in  defeating  IR  countermeasures  such  as  smoke,  camouflage,  and  flares. 
The  MDSS  program  has  supported  research  aimed  at  producing  large-format,  staring,  dual-band  FPAs. 
Both  quantum  well  infrared  photodetector^  (QWIP)  and  mercury-cadmium  telluride  (HgCdTe  or  MCT) 
technologies  were  investigated.  This  paper  presents  the  results  for  the  QWIP  devices. 

The  quantum  well  infrared  photodetector  is  made  with  GaAs  and  other  III-V  semiconductors,  and 
has  been  put  forward  as  an  alternative  to  the  incumbent  MCT  devices  for  dual-band  infrared  FPAs.  The 
QWIP  detector  structure  has  the  advantage  over  an  MCT  detector  because  the  spectral  response  is  rela¬ 
tively  narrow  which  means  that  a  detector  designed  for  the  8  to  10- pm  wavelength  band  detects  light  only 
in  that  band  with  little  or  no  “spectral  crosstalk”  with  a  detector  in  the  3  to  5 -pm  band.  QWIPs  have  been 
designed  and  demonstrated  to  operate  effectively  at  peak  wavelengths  as  short  as  3  pm  to  longer  than  16 
pm.  Dual-band  detector  structures  present  no  serious  materials  growth  issues  because,  for  the  most  part, 
the  operating  wavelengths  are  determined  by  the  thicknesses  of  the  layers,  a  feature  easily  controlled  with 
molecular  beam  epitaxy  (MBE).  In  addition,  the  material  growth  and  processing  technology  are  more 
mature  in  the  III-V  material  systems  than  II- VI  materials  like  MCT,  allowing  for  high  operability  of  FPAs 
with  uniform  pixel-to-pixel  response,  which  is  the  main  performance  limitation  of  modem  IR  FPAs. 

The  principal  disadvantages  of  QWIPs  are  their  low  single-pixel  quantum  efficiency  relative  to 
MCT  and  relatively  high  dark  currents  at  temperatures  near  that  of  liquid  nitrogen  (77  K).  The  high  dark 
current  at  77  K  necessitates  cooling  to  temperatures  around  65  K  or  below  to  obtain  background-limited 
performance  (BLIP)  in  LWIR  QWIPs.  Nevertheless,  excellent  IR  imagery  has  been  demonstrated  with 
large  format  (640  x  480  pixels)  FPAs. 

We  will  describe  the  fabrication  and  test  results  of  a  256  x  256  pixel-registered  dual-band  FPA 
designed  to  operate  simultaneously  in  the  MWIR  and  LWIR  spectral  regions.  We  will  also  present  im¬ 
agery  taken  in  the  field  of  targets  of  military  significance  like  armored  vehicles,  tmcks,  and  helicopters. 
We  have  applied  image  fusion  algorithms  to  the  dual-band  imagery.  The  results  indicate  that  high-quality 
IR  imagery  can  be  obtained  in  both  bands  at  operating  temperatures  lower  than  65  K.  We  will  then  point 
out  some  of  the  shortcomings  of  this  initial  demonstration  and  recommend  solutions. 


H.  Pollehn  and  J.  Aheam,  “Multi-Domain  Smart  Sensors,”  Proceedings  of  the  SPIE,  Infrared  Technology  and  Ap¬ 
plications  XXV,  Vol.  3698,  Orlando,  FL  (1999). 

^  K.  K.  Choi,  The  Physics  of  Quantum  Well  Infrared  Photodetectors,  World  Press,  Singapore,  (1997). 


2.0  FPA  FABRICATION 


2.1  Detector  Array 

Details  of  the  growth  and  processing  of  the  dual-band  QWIP  FPAs  used  in  this  work  have  been 
published  in  Goldberg,  et  al/  Figure  1  shows  the  device  structure  schematically  and  Figure  2  is  a  scan¬ 
ning  electron  micrograph  of  several  pixels  of  a  dual-band  QWIP  FPA  prior  to  indium  bump  deposition. 
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Figure  1.  Schematic  representation  of  the  dual-band  QWIP  detector  structure. 

The  pixels  were  square  with  a  40- pm  pitch  in  both  the  x-  and  y-directions.  The  space  between 
pixels  was  less  than  2  pm  leading  to  a  fill  factor  of  greater  than  84%  in  the  MWIR  detectors  (bottom 
layer)  and  greater  than  77%  for  the  LWIR  detectors  (top  layer). 


Detector  common 


LWIR 

=LWIR  QWIP 


^MmWIR  QWIP 


MWIR  bias 


Etch-stop 

layers 


Figure  2.  Section  of  two-color  QWIP  FPA  prior  to  indium  bump  deposition,  showing  high  optical  fill 
factor,  vertical  sidewalls,  and  metal  lines  to  each  ohmic-contact  layer. 


^  A.  Goldberg,  S.  Wang,  M.  Sundaram,  P.  Uppal,  M.  Winn,  G.  Milne  and  M.  Stevens,  “Dual  Band  MWIR/LWIR 
Focal  Plane  Array  Test  Results,”  Proceedings  of  the  1999  IRIS  Detector  Specialty  Group  Meeting  on  Detectors,  p. 
87,  ERIM  (2000). 


2.2  Read-out  Integrated  Circuit  (ROIC) 


Lockheed  Martin  Electronics  and  Missile  Systems  designed  the  ROIC.  The  simultaneously  inte¬ 
grating  and  windowing  ROIC  was  fabricated  by  using  a  0.5-|Lim  complementary  metal  oxide  semicon¬ 
ductor  (CMOS)  process.  It  is  designed  to  consume  less  than  50  mW  power  with  a  5-V  supply,  uses  a 
switched-capacitor  frame  snapshot  architecture  capable  of  frame  rates  up  to  180  frames/s,  and  can  provide 
variable  bias  for  each  color.  The  capacitance  of  the  integration  capacitor  can  be  selected  with  a  serial 
word  command.  In  low-gain  mode,  the  capacitance  of  the  charge  well  is  1  pF,  and  in  high-gain  mode,  the 
capacitance  is  0.1  pF  (well  capacity  2.5  x  10^  and  2.5  x  10^  electrons,  respectively  for  VDD  =  5  V).  This 
ROIC  has  also  been  used  for  dual-band  LWIR/LWIR  and  MWIR/MWIR  imaging  FPAs.^  In  the  high 
gain  mode,  a  switched-capacitor  filter,  used  to  reduce  output  noise,  is  enabled.  The  implementation  of 
this  filter  has  been  shown  to  reduce  the  FPA’s  ability  to  respond  to  transient  signals.*^ 

The  ROIC  provides  bias  to  the  detectors  in  the  following  way:  The  ohmic-contact  layer  between 
the  MWIR  and  LWIR  QWIP  structures  is  made  common  for  all  pixels.  This  detector-common 
(DETCOM)  voltage  is  then  held  above  the  main  power  bias  for  the  ROIC  chip  (VDD).  Bias  levels  on  the 
LWIR  and  MWIR  sections  are  maintained  using  separate  bias  supplies  connected  to  the  top  and  bottom 
ohmic-contact  bumps  of  the  detector  pixels,  respectively.  The  detector  bias  voltages  need  to  be  less  than 
VDD.  In  this  way,  the  LWIR  structure  is  biased  such  that  the  electrons  flow  toward  the  substrate,  and  in 
the  MWIR  portion  of  the  detector,  the  electrons  flow  away  from  the  substrate. 

3.0  RESULTS  AND  DISCUSSION 

3.1  Laboratory  Test  Results 

The  tests  on  the  dual-band  FPA  were  done  at  ARL  with  the  FPA  installed  in  a  liquid  helium  flow¬ 
through  cryostat  (Lakeshore  MTD-150).  The  input  clock  and  bias  signals,  as  well  as  the  output  pixel 
data,  were  provided  and  collected  by  a  configurable  FPA  drive  electronics  unit  made  by  SE-IR  Corpora¬ 
tion.  The  pixel  signals  were  digitized  using  12-bit  analog-to-digital  converters  (A/DC)  capable  of  data 
rates  up  to  12  megapixels/s.  The  cryostat  was  equipped  with  an  internal,  cooled  aperture  wheel  allowing 
for  operation  of  the  FPA  under  a  wide  range  of  background  flux  conditions  (including  zero-background). 

Table  1  shows  the  bias  values  used  to  run  the  FPA.  The  frame  rate  used  for  tests  was  typically  60 
Hz  when  viewing  the  blackbody  source.  This  allowed  for  integration  times  as  short  as  80  ps  and  as  long 
as  16  ms.  For  the  dark-current  tests,  frame  rates  as  low  as  1  Hz  and  integration  times  as  long  as  800  ms 
were  used. 

Under  typical  tactical  background  conditions  (background  temperature  -300  K,  f/2.0  to  f/3.0)  the 
photon  flux  in  the  MWIR  portion  of  the  spectrum  covered  by  this  FPA  is  approximately  10^  pho- 
tons/cm^/s  while  that  for  the  LWIR  band  is  about  one  order  of  magnitude  higher.  Therefore,  the  MWIR 
portion  of  the  array  was  nearly  always  run  in  the  “high-gain”  mode  with  charge  well  capacitance  of  0.1 
pF  while  the  LWIR  portion  was  run  in  “low-gain”  mode  with  its  charge  well  capacitance  being  1.0  pF. 


^  G.  Milne,  K.  Reiff,  M.  West,  R.  Martin,  T.  Faska,  M.  Sundaram,  M.  Taylor,  R.  Williams,  and  D.  Hayden,  “Ad¬ 
vances  in  QWIP  FPA  (2-Color  and  1 -Color)  Technology,”  Proceedings  of  the  1999  Meeting  of  the  IRIS  Specialty 
Group  on  Passive  Sensors,  ERIM  (1999). 

^  J.  Hubbs,  D.  Arrington,  M.  Gramer,  and  G.  Doyle,  “Radiometric  Characterization  of  a  Two  Color  Long/Long 
Wavelength,  QWIP  Focal  Plane  Array  Under  Low  Background  Conditions,”  Proceedings  of  thel999  Meeting  of  the 
MSS  Specialty  Group  on  Infrared  Detectors,  p.  105,  ERIM  (2000). 


Table  1.  Bias  functions  applied  to  the  dual-band  FPA  and  their  values. 


Bias  name 

Function 

Values  used  (V) 

VDD 

Main  power  voltage  for  the  ROIC. 

5.00 

DETCOM 

Detector  common  voltage  applied  to  the  intermediate  oh¬ 
mic-contact  layer. 

5.50-6.50 

DIBIAS  1 

Provide  bias  to  the  “top”  ohmic-contact  layer.  Used  to  bias 
the  LWIR  device. 

3.40-4.40 

DIBIAS2 

Provide  bias  to  the  “bottom”  ohmic-contact  layer.  Used  to 
bias  the  MWIR  device. 

2.00-3.00 

The  bias  actually  applied  to  the  MWIR  and  LWIR  sections  of  the  detector  array  depends  on  all  the  values 

in 


Table  1.  For  the  LWIR  detector,  the  bias  is  given  by 

L  WBias  =  {DETCOM  -  VDD)  +  (3.90-  DIBIASX)  volts  ( 1 ) 

and  the  bias  on  the  MWIR  detector  is 

MWBias  =  {DETCOM  -  VDD)  +  (3.90-  DIBIAS2)  volts.  (2) 

3.1.1  Spectral  Response 

The  spectral  response  of  the  FPA  was  measured  by  forming  an  image  of  the  output  slit  of  a  grat¬ 
ing  monochrometer  and  recording  the  changes  in  the  outputs  of  the  pixels  as  a  function  of  wavelength. 
Figure  3  shows  the  results  of  these  measurements.  The  response  spectra  measured  for  the  two  outputs  of 
the  FPA  are  slightly  different  from  those  reported  in  figure  3  of  our  previous  work.^  The  earlier  data  were 
measured  on  single-pixel  devices  showed  the  peak  of  the  MWIR  response  to  be  5.1  pm  with  cut-on  and 
cut-off  wavelengths  of  4.7  pm  and  5.4  pm,  respectively. 
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Figure  3.  Normalized  response  spectra  for  the  MWIR  and  LWIR  sections  of  the  dual-band  QWIP  FPA. 

3.1.2  Dark  Current  and  BLIP  Temperature 

For  dark-current  measurements,  we  placed  the  zero  field-of-view  cold  stop  in  front  of  the  FPA. 
The  current  was  measured  by  integrating  charge  from  the  detectors  for  as  long  a  time  as  necessary  to  fill 
the  integration  capacitor  approximately  halfway  to  saturation.  A  number  of  frames  (typically  50)  were 
then  collected  and  averaged.  The  current  was  then  calculated  using 

I  =  -NqG,^  (3) 

T 

where  A  is  the  number  of  counts  in  the  A/D  (from  0  to  4095),  C/  is  the  capacitance  of  the  integration  ca¬ 
pacitor,  Gad  is  the  gain  of  the  A/D  (V/count),  and  ris  the  integration  time.  Similar  data  were  collected 
with  the  shutter  open  (f/2.5  aperture)  to  measure  the  sum  of  the  dark  current  and  photocurrent.  Figure  4 
shows  the  results  of  these  measurements.  If  we  define  the  background-limited  performance  (or  BLIP) 
temperature  as  the  operating  temperature  at  which  the  dark  current  and  the  background-induced  photocur¬ 
rent  are  equal,  then  the  LWIR  array  is  background  limited  below  approximately  65  K  and  the  MWIR  ar¬ 
ray  is  background  limited  below  approximately  95  K.  The  data  agree  well  with  the  values  predicted  from 
tests  performed  on  single-pixel  detectors. 

The  dark-current  data  follow  the  expected  exponential  dependence  on  temperature  with  thermal 
activation  energies  of  226  meV  for  the  MWIR  structure  and  120  meV  for  the  LWIR  part  of  the  array. 
These  energies  correspond  to  wavelengths  of  5.48  pm  and  10.32  pm  for  the  MWIR  and  LWIR  structures, 
respectively.  Figures  5  and  6  show  the  histogram  and  spatial  distributions  of  the  thermal  activation  ener¬ 
gies  for  the  LWIR  and  MWIR  portions  of  the  FPA.  The  mean  activation  energies  were  119.3±3.0  meV 
and  226.3±9.97  meV,  corresponding  to  nonuniformities  in  activation  energy  of  2.5%  and  4.4%  for  the 
LWIR  and  MWIR  arrays,  respectively.  However,  since  the  dark  current  depends  exponentially  on  the 
activation  energy,  these  small  variations  in  activation  energy  caused  the  dark  current  to  vary  by  more  than 
a  factor  of  two  across  the  FPA.  The  nonuniformity  of  the  activation  energy  (and  therefore,  the  dark  cur¬ 
rent)  is  a  major  limiting  factor  in  operating  the  QWIP  FPA  in  the  dark-current  limit.  At  temperatures 


above  the  BLIP  temperature,  the  large  variation  in  dark  current  limits  the  gain  that  can  be  applied  to  the 
output  signals  to  a  low  value.  Under  these  conditions,  the  span  of  output  values  covers  the  entire  range  of 
the  A/DC. 

To  operate  effectively  at  high  temperatures,  the  variation  in  dark  current  should  be  no  more  than 
that  of  the  photocurrent  (5  percent  or  less).  For  a  small  change  in  activation  energy  SEa,  the  correspond¬ 
ing  change  in  dark  current  is  =  —dE^jkT,  where  T  is  the  operating  temperature.  To  achieve  a 

dark-current  uniformity  of  95  percent  or  better  at  77  K,  the  activation  energy  must  vary  by  no  more  than 
0.33  meV  or  0.275  percent.  The  observed  variation  in  activation  energy  is  an  order  of  magnitude  above 
this  level. 


The  images  in  Figures  5  and  6  show  that  the  activation  energy  decreases  monotonically  from  left 
to  right  across  the  FPA  for  both  the  LWIR  and  MWIR  arrays.  This  behavior  is  most  likely  indicative  of 
variations  in  the  material  growth  parameters  (growth  rates  and/or  alloy  composition)  across  the  chip.  A 
substantial  reduction  in  such  nonuniformities  is  one  factor  that  may  allow  for  operation  of  QWIPs  at 
higher  temperatures. 
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Figure  4.  Mean  dark  current  and  total  current  for  the  MWIR  and  LWIR  portions  of  the  dual-band  FPA. 
The  background  temperature  was  298  K  (25°  C).  The  bias  levels  were  1.0  V  for  the  LWIR  and  2.0  V  for 
the  MWIR. 


Figure  5.  Histogram  (a)  and  image  (b)  of  the  distribution  of  thermal  activation  energies  for  the  LWIR 
pixels  of  the  dual-band  FPA.  The  mean  activation  energy  is  1 19.3±3.0  meV. 


Figure  6.  Histogram  (a)  and  image  (b)  of  the  distribution  of  thermal  activation  energies  for  the  MWIR 
pixels  of  the  dual-band  FPA.  The  mean  activation  energy  is  226.3±9.97  meV. 


3.1.3  Imaging  Performance,  Operability,  and  Uniformity 

The  noise  equivalent  temperature  difference  (NEAT)  was  calculated  from  uncorrected  imagery  of 
an  extended-area  blackbody  at  different  temperatures.  The  raw  data  consisted  of  many  (usually  64)  con¬ 
secutive  frames  collected  into  a  single  file  with  a  framegrabber  built  into  the  PC  that  operates  the  IR  cam- 


era  electronics.  The  data  were  then  analyzed  using  the  NVESD  3-D  noise  model^  to  obtain  the  various 
components  of  NEAT.  The  data  was  taken  at  blackbody  temperatures  covering  a  12°C  range  in  tem¬ 
perature  from  20°C  to  32°C;  the  calibration  temperatures  were  defined  as  22°C  and  30°C.  The  data  from 
each  of  the  two  bands  was  analyzed  separately.  Data  were  taken  under  a  wide  variety  of  operating  condi¬ 
tions  (bias  levels)  and  at  device  temperatures  ranging  from  50  to  1 10  K. 

We  found  that,  under  BLIP  conditions,  we  could  make  the  FPA  performance  relatively  independ¬ 
ent  of  the  bias  on  the  detectors  by  shortening  or  lengthening  the  integration  time  as  needed  to  get  the  sig¬ 
nals  to  cover  most  of  the  A/D  range  for  a  reasonable  change  in  background  temperature  (-10  °C).  Above 
the  BLIP  temperature,  the  large  variation  in  output  current  made  it  difficult  to  find  good  operating  condi¬ 
tions.  In  particular,  in  the  temperature  range  between  65  and  70  K,  the  short  integration  times  mandated 
by  the  relatively  high  dark  currents  in  the  LWIR  part  of  the  FPA  also  degraded  the  performance  of  the 
MWIR  portion  of  the  device.  At  temperatures  of  70  K  and  above,  it  was  impossible  to  operate  the  LWIR 
array  and  so  the  operating  conditions  were  optimized  for  best  MWIR  performance. 

Figure  7  shows  the  3-D  noise  components  of  NEAT  for  the  LWIR  (a)  and  MWIR  (b)  portions  of 
the  FPA  at  an  operating  temperature  of  60  K.  The  integration  time  used  was  8.0  ms,  the  LWIR  bias  was 
1.0  V,  and  the  MWIR  bias  was  2.0  V.  The  temporal  NEAT  (Ntvh)  was  found  to  be  approximately  the 
same  in  both  bands  at  about  32  mK.  For  the  operating  conditions  described  above.  Figure  8  shows  the 
histogram  and  image  of  the  conversion  efficiency  of  the  LWIR  array  and  Figure  9  shows  those  for  the 
MWIR  array.  The  imaging  performance  properties  are  summarized  in  Table  2  below. 


Table  2.  Performance  properties  of  the  dual-band  MWIR/LWIR  FPA  at  T=60  K. 


Band 

Bias 

(V) 

Mean  Temporal 
NEAT  (mK) 

Mean  Conversion 
Efficiency  (%) 

Pixels  with  Re¬ 
sponse  Outside 
±10%  of  Mean 

Pixels  with  Re¬ 
sponse  Outside 
±20%  of  Mean 

LWIR 

1.00 

34 

2.61±0. 119  (±4.56%) 

229  (0.35%) 

121  (0.18%) 

MWIR 

2.00 

30 

1.91±0. 130  (±6.82%) 

6346  (9.7%) 

170  (0.26%) 

The  magnitude  of  the  conversion  efficiency  as  well  as  the  pixel-to-pixel  uniformity  of  the  re¬ 
sponse  are  significantly  better  in  the  LWIR  than  in  the  MWIR.  In  both  bands,  the  operability  of  the  array 
is  excellent  with  greater  than  99.7  percent  of  the  pixels  having  conversion  efficiency  values  within  20 
percent  of  the  mean.  The  fixed-pattern  noise  is,  as  expressed  by  the  value  of  N^h  in  the  3-D  noise,  is  sig¬ 
nificantly  larger  in  the  MWIR  data  as  opposed  to  that  for  the  LWIR  array.  This  is  probably  due  to  the 
higher  nonuniformity  of  response  in  the  MWIR  array  relative  to  that  of  the  LWIR  array. 
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Figure  7.  3-D  noise  components  as  a  function  of  background  temperature  for  the  LWIR  array  (a),  and  the 
MWIR  array  (b)  at  a  device  temperature  of  60  K.  The  integration  time  used  was  8.0  msec,  the  LWIR  bias 
was  1 .0  V,  and  LWIR  bias  was  2.0  V. 


Figure  8.  Histogram  (a)  and  grayscale  image  (b)  of  the  conversion  efficiency  (quantum  efficiency-gain 
product)  for  the  LWIR  part  of  the  FPA. 


Figure  9.  Histogram  (a)  and  grayscale  image  (b)  of  the  conversion  efficiency  (quantum  efficiency-gain 
product)  for  the  MWIR  part  of  the  FPA. 


Figure  10  shows  the  results  of  measurements  of  temporal  NEAT  as  a  function  of  operating  tem¬ 
perature.  Below  65  K,  the  arrays  from  both  bands  had  similar  values  of  NEAT  (between  30  and  40  mK). 
However,  once  the  device  temperature  was  higher  than  the  BLIP  temperature,  the  NEAT  rose  dramati¬ 
cally  in  both  bands.  The  NEAT  performance  of  the  MWIR  array  degraded  because  short  integration  times 
were  needed  to  keep  the  LWIR  portion  of  the  FPA  from  saturation  (it  was  not  possible  to  control  the  inte¬ 
gration  times  for  each  band  independently).  At  temperatures  above  70  K,  the  operating  conditions  were 
optimized  for  the  MWIR  band.  The  NEAT  was  approximately  30  mK  for  temperatures  less  than  90  K;  at 
temperatures  above  90  K,  the  dark  current  becomes  dominant  in  the  MWIR  device,  thus  causing  the 
NEAT  to  rise  sharply. 
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Figure  10.  Mean  temporal  NEAT  (Ntvh)  as  a  function  of  operating  temperature  for  the  dual-band  FPA. 
The  circles  and  squares  are  the  values  of  NEAT  measured  when  operating  conditions  were  optimized  for 
running  both  bands  simultaneously,  the  stars  show  the  values  of  NEAT  measured  when  the  conditions 
were  optimized  for  the  MWIR  array  alone.  The  lines  are  simply  guides  to  the  eye. 

3.2  Image  Fusion 

The  images  acquired  in  the  field  tests  performed  with  the  dual-band  QWIP  FPA  were  subjected  to 
a  color  fusion  algorithm,  developed  at  the  Naval  Research  Laboratory  (NRL),  in  which  pixel  values 
from  the  LWIR  and  MWIR  bands  are  assigned  to  color  opponents  such  as  red-cyan,  blue-yellow,  or 
green-magenta.  Figure  11  illustrates  this  color  fusion  scheme  using  the  red-cyan  color  opponents.  Each 
pixel  in  the  LWIR  image  is  assigned  a  red  value  and  each  pixel  in  the  MWIR  image  is  assigned  a  cyan 
value  (i.e.,  equal  values  of  blue  and  green).  For  eight-bit  color,  the  pixel  values  range  from  0  to  255. 
Objects  in  the  image  with  high  brightness  values  in  both  bands  will  appear  white;  those  with  low  bright¬ 
ness  values  in  both  bands  will  appear  black.  Objects  with  a  high  pixel  value  in  the  LWIR  band  and  a  low 
value  in  the  MWIR  band  will  appear  red,  and  objects  with  a  low  pixel  value  in  the  LWIR  band  and  a  high 
pixel  value  in  the  MWIR  band  will  appear  cyan. 


D.  Scribner,  J.  Schuler,  P.  Warren,  M.  Satyshur,  M.  Kmer,  “Infrared  Color  Vision:  Separating  Objects  from 
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Figure  11.  Red-cyan  color  scheme  for  fusing  MWIR  and  LWIR  images.  The  MWIR  image  is  mapped  to 
shades  of  cyan  and  the  LWIR  image  is  mapped  to  shades  of  red.  Pixels  with  approximately  equal  values 
in  the  MWIR  and  LWIR  will  lie  along  the  diagonal  of  the  color  diagram  and  will  be  represented  by 
shades  of  gray. 

In  this  scheme,  bands  in  which  the  background  and  targets  are  highly  correlated  will  yield  fused 
images  with  little  color  contrast  (the  pixel  data  will  lie  roughly  along  the  diagonal,  [0,0]  to  [255,255],  of 
the  plot  and  the  image  will  appear  as  shades  of  gray).  Bands  in  which  they  are  weakly  correlated  will 
yield  maximum  color  contrast  (the  pixel  data  will  be  spread  out  in  a  direction  orthogonal  to  the  diagonal). 
Where  the  background  is  highly  correlated  and  the  target  is  only  slightly  different,  the  color  contrast  can 
be  enhanced  by  performing  a  principal  component  (PC)  transformation,  normalizing  the  data  along  the 
PC  directions  (thereby  stretching  the  data  to  fill  the  available  color  space),  then  performing  the  inverse 
transform  back  to  the  original  color-opponent  space. 

3.3  Field  Imaging  Results 

The  dual-band  FPA  was  installed  in  a  portable  camera  system  and  taken  to  the  field  to  image  tar¬ 
gets  of  military  significance.  Tests  were  conducted  at  Ft.  A.  P.  Hill  and  at  the  Aberdeen  Proving  Ground 
(APG)  during  1999.  In  both  of  these  tests  the  imaging  optical  system  was  a  100-mm  focal  length  f/3.0 
dual-band  lens  made  by  Diversified  Optical  Products,  Inc.  With  this  lens,  the  instantaneous  field  of  view 
(IFOV)  was  0.4  mrad  and  the  total  FOV  was  5.86°  in  both  altitude  and  azimuth.  Given  the  relatively 
short  focal  length  of  the  optics  (and  therefore  large  FOV)  it  was  difficult  to  resolve  targets  at  ranges 
longer  than  1  km.  Nevertheless,  it  is  possible  to  see  the  potential  advantages  of  dual-band  imaging  with 
the  images  presented  below.  In  addition,  the  placement  of  the  MWIR  spectral  peak  at  5.0  pm  causes  the 
FPA’s  response  spectrum  to  miss  much  of  the  MWIR  transmission  window.  This  caused  the  MWIR  im¬ 
agery  to  be  of  lower  contrast  than  that  of  the  LWIR  under  all  ambient  conditions  observed  in  these  field 
tests.  In  addition,  the  relatively  low  conversion  efficiency  of  the  detectors  necessitated  the  use  of  an  inte¬ 
gration  time  of  approximately  10  ms. 


3.3.1  Armored  Vehicles 


Figure  12  shows  an  M60  tank  moving  along  a  dirt  and  gravel  road.  The  exhaust  plume  is  more 
prominent  in  the  MWIR  image  than  in  the  LWIR  image.  In  addition,  the  concrete  slab  in  the  foreground 
is  very  bright  in  the  MWIR  image,  but  relatively  dark  in  the  LWIR  image. 


Figure  12.  Image  of  an  M60  tank  taken  with  the  dual-band  MWIR/LWIR  QWIP.  The  left-hand  image 
(a)  is  the  LWIR  and  the  right  image  (b)  is  MWIR.  Note  that  the  exhaust  plume  of  the  tank  and  concrete 
slab  are  more  prominent  in  the  MWIR  image. 

Figure  13  shows  an  M60  tank  and  an  M2  Bradley  fighting  vehicle  at  a  range  of  approximately 
200  m.  This  image  was  taken  just  before  noon  on  a  summer  day.  The  M60  has  kicked  up  a  large  dust 
trail  that  shows  prominently  in  the  LWIR  image,  as  seen  in  Figure  13(a),  but  much  less  so  in  the  MWIR 
image  shown  in  Figure  12(b).  Figure  13(c)  shows  the  simple  difference  image  of  the  two  single-color 
images. 


It  is  interesting  to  note  that  in  image  data  collected  during  the  MDSS  field  test  in  1998  (in  which 
separate  MWIR  and  LWIR  cameras  were  boresighted  to  give  a  simulated  dual-band  FLIR  output)  the  dust 
plume  from  a  moving  vehicle  appeared  more  prominent  in  the  MWIR  than  the  LWIR.  This  result  has 
been  reported  elsewhere. The  results  of  an  application  of  the  color  fusion  algorithm  described  above  of 
the  MWIR  and  LWIR  image  data  from  Figure  13  and  from  the  separate-camera  image  data  are  shown  in 
Figures  14  (a)  and  (b),  respectively.  The  data  that  were  used  to  generate  the  images  in  Figure  14  were 
acquired  at  the  same  test  range  at  the  same  season  (late  July),  at  approximately  the  same  time  of  day 
(early  afternoon)  and  under  very  similar  meteorological  conditions. 
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Figure  13.  Simultaneous  dual-band  image  of  an  M60  tank  and  an  M2  Bradley  fighting  vehicle.  The 
LWIR  image  (a)  clearly  shows  a  dust  plume  behind  the  tank  that  is  barely  visible  in  the  MWIR  image  (b). 
In  the  MWIR  image,  the  engine  exhaust  of  the  M2  can  be  seen  more  clearly  than  in  the  LWIR  image. 
The  fused  difference  image  (LWIR  —  MWIR)  is  shown  in  (c);  objects  with  large  LWIR  signatures  (rela¬ 
tive  to  MWIR)  are  bright  and  those  with  large  MWIR  signatures  are  dark. 


Figure  14.  (a)  Color  fusion  image  of  the  LWIR  and  MWIR  images  shown  in  Figure  13.  The  dust  plume 
from  the  M60  is  most  prominent  in  the  LWIR  data  and  therefore  show  appears  red  in  the  fused  image. 
Conversely,  the  exhaust  plumes  from  the  vehicles  emit  strongly  in  the  MWIR  band  and  are  therefore 
shown  here  as  blue,  (b)  Color  image  fusion  of  MWIR  and  LWIR  data  from  separate  cameras.  This  im¬ 
age  was  acquired  as  part  of  the  MDSS  field  test  in  July,  1998.  The  dust  plume  created  by  the  motion  of 
the  M2  was  more  prominent  in  the  MWIR  image  and  is  therefore  shown  as  cyan  in  this  image. 

Figure  15  shows  a  series  of  images  of  an  MIA  1  tank  moving  down  a  gravel  road  in  early  even¬ 
ing.  The  left  image  is  LWIR,  the  center  image  is  MWIR  and  on  the  right  is  the  color  fusion  image.  In  the 
tank’s  wake,  a  blast  of  warm  air  that  slightly  heated  the  leaves  on  the  trees  at  the  side  of  the  road.  This 
heating  shows  up  best  in  the  LWIR  image.  Therefore,  the  fused-color  image  displays  the  heated  leaves  as 
shades  of  red.  As  the  tank  gets  closer,  it  is  easier  to  see  the  MWIR  signature  (shades  of  blue)  of  the  en¬ 
gine  exhaust. 
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Figure  15.  Images  of  an  MIA  1  tank  approaching  the  camera.  The  bright  areas  behind  the  tank  are  tree 
leaves  that  were  heated  by  the  exhaust  gases  of  the  tank.  The  exhaust  plume,  which  predominantly  radi¬ 
ates  in  the  MWIR,  becomes  prominent  as  the  vehicle  gets  closer.  The  body  of  the  tank  itself  appears  to 
radiate  most  strongly  in  the  LWIR  and  therefore  appears  red  in  the  color  fused  images. 

3.3.2  Trucks 

Figure  16  is  a  dual-band  image  of  a  convoy  of  3  military  trucks  moving  down  a  gravel  road  at 
night;  the  range  to  the  targets  was  approximately  500  m.  The  vehicles  imaged  were:  a  HMMWV,  a  Rus¬ 
sian  Zill-131  truck  and  an  M3  5  truck.  Figure  17  shows  imagery  of  these  targets  at  close  range  (approxi¬ 
mately  100  m).  The  MWIR  imagery  of  the  Zill  truck  (Figure  17c)  contains  dark  vertical  lines  extending 
from  hot  objects  on  the  target.  These  artifacts  do  not  appear  in  the  LWIR  image  and  are  most  likely 
caused  by  the  switched-capacitor  filter  present  in  high-gain  mode  of  the  ROIC. 
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Figure  16.  Dual-band  image  of  three  military  trucks  at  a  distance  of  approximately  500  m. 


LWIR 

r  ^ 

MWIR 

1 

Color  Fused 

(a) 

r  ■  ^ 

(b) 

(C) 

Figure  17.  Dual-band  images  of  a  HMMWV  (a),  a  Zill-131  (Russian)  truck  (b),  and  an  M35  truck.  The 
left  hand  images  are  LWIR,  the  center  images  are  MWIR,  and  the  right  hand  images  are  the  results  of 
color  fusion. 


3.3.3  Helicopters 


Figure  18  shows  dual-band  images  of  different  helicopters  acquired  at  night  at  relatively  close 
range  (less  than  500  m).  The  smearing  of  the  signature  of  the  rotor  blades  can  be  clearly  seen  in  the 
MWIR  image;  it  is  not  nearly  as  apparent  in  the  LWIR  image.  In  addition,  the  hot  engines  create  dark 
vertical  line  artifacts  in  the  MWIR  image  that  are  not  seen  in  the  LWIR  image  (not  shown  in  the  images 
in  Figure  18).  Both  of  these  artifacts  are  caused  by  the  switched-capacitor  filter  enabled  for  the  MWIR 
portion  of  the  FPA  (due  to  the  need  to  run  at  high  gain). 
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Figure  18.  Dual-band  images  of  three  different  helicopters:  Blackhawk  (a),  Cobra  (b),  and  OH-58  (c). 

4.0  SUMMARY  AND  CONCLUSIONS 

We  have  presented  the  results  of  laboratory  and  field  tests  of  a  pixel-registered,  simultaneously 
integrating  dual-band  FPA  operating  in  the  MWIR  (peaked  at  5.0  pm)  and  in  the  LWIR  (peaked  at  8.6 
pm)  made  using  QWIP  technology.  The  laboratory  tests  indicated  that  the  array  had  excellent  operability 


(>99.7  percent)  and  good  imaging  performance  in  both  bands  for  temperatures  less  than  65  K.  The  tem¬ 
poral  NEAT  was  found  to  be  approximately  32  mK  in  both  bands  below  65  K  and  approximately  30  mK 
for  the  MWIR  band  for  temperatures  less  than  90  K.  The  field  performance  of  the  dual-band  QWIP  FPA 
was  very  good,  given  the  parameters  of  the  optical  system.  Specific  issues  in  the  level  of  performance 
(i.e.,  conversion  efficiency,  spectral  placement)  of  the  individual  pixels  need  to  be  addressed.  However, 
the  operability  and  uniformity  of  the  FPA  in  both  bands  is  quite  impressive.  Single-pixel  results  on  dual¬ 
band  QWIP  material  lattice-matched  to  an  InP  substrate  that  was  grown  at  ARE  indicate  that  these  issues 
are  not  insurmountable.^^  Dual-band  QWIP  FPAs  with  considerably  higher  conversion  efficiencies  than 
those  quoted  here  in  both  the  MWIR  and  LWIR  bands  are  possible.  The  smearing  of  imagery  of  fast- 
moving  targets  in  the  MWIR  is  a  direct  result  of  a  sub-optimal  implementation  of  a  switched-capacitor 
filter  in  the  high-gain  mode  of  the  ROIC.  Others  have  observed  a  behavior  very  much  like  dielectric  re¬ 
laxation  in  QWIPs  at  low  temperatures  and  backgrounds.  We  believe  that  under  the  conditions  in  which 
this  FPA  was  operated,  the  dielectric  relaxation  would  have  affected  the  MWIR  imagery  much  less  than 
what  we  observed.  A  minor  modification  to  the  ROIC  would  almost  completely  eliminate  the  observed 
image  smearing.  If  the  MWIR  conversion  efficiency  was  10%  (as  opposed  to  the  measured  2%),  the  ad¬ 
ditional  photocurrent  would  short  out  any  residual  dielectric  relaxation  effects,  thus  allowing  for  sharp 
MWIR  images.  We  expect  to  take  delivery  of  an  improved  dual-band  QWIP  FPA  later  this  year. 

It  appears  from  the  field  imagery  that  the  differences  in  the  MWIR  and  LWIR  signatures  of  the 
targets  studied  were  not  nearly  as  significant  as  those  of  the  disturbances  that  the  targets  made  to  their 
immediate  environment.  This  would  include  the  exhaust  plumes  from  engines,  dust  kicked  up  from  the 
ground,  and  heat  transferred  from  the  targets  to  their  surroundings.  However,  even  images  taken  under 
similar  conditions  can  yield  vastly  different  results.  The  two  images  shown  in  Figure  14  were  both  taken 
in  mid-summer  at  the  same  test  site  at  nearly  the  same  time  of  day  and  under  very  similar  meteorological 
conditions.  However,  in  Figure  14  the  dust  kicked  up  by  the  moving  vehicle  emits  and/or  reflects  more 
LWIR  light  than  MWIR,  while  in  Figure  15,  the  dust  emitted/refiected  more  MWIR  than  LWIR  light. 
Evidently,  subtle  differences  in  the  conditions  can  have  a  major  impact  on  the  observed  imagery. 

One  of  the  principal  motivations  behind  dual-band  IR  imaging  systems  has  been  the  supposed 
ability  to  pick  out  targets  in  a  cluttered  environment  better  than  a  single-color  system  or  similar  perform¬ 
ance.  In  our  field  tests  reported  here,  none  of  the  targets  were  embedded  in  any  sort  of  a  cluttered  setting. 
Therefore,  we  find  it  difficult  to  conclude  from  these  data  whether  this  MWIR/LWIR  dual-band  camera 
will  be  able  to  fulfill  the  promise  of  enhanced  target  detection/recognition  performance  in  such  situations. 
However,  in  a  previous  field  test  using  separate  MWIR  and  LWIR  cameras,  the  ATR  performance  was 
enhanced  significantly  when  imagery  from  both  cameras  was  used  over  that  for  single  color  data.^^ 
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